生物医学机器阅读理解（生物医学MRC）旨在理解复杂的生物医学叙事，并协助医疗保健专业人员从中检索信息。现代神经网络的MRC系统的高性能取决于高质量的大规模，人为宣传的培训数据集。在生物医学领域中，创建此类数据集的一个至关重要的挑战是域知识的要求，引起了标记数据的稀缺性以及从标记的通用（源）域转移学习到生物医学（目标）域的需求。然而，由于主题方差，通用和生物医学领域之间的边际分布存在差异。因此，从在通用域上训练的模型到生物医学领域的模型直接转移学会的表示可能会损害模型的性能。我们为生物医学机器阅读理解任务（BioAdapt-MRC）提供了基于对抗性学习的域适应框架，这是一种基于神经网络的方法，可解决一般和生物医学域数据之间边际分布中的差异。 Bioadapt-MRC松弛了生成伪标签的需求，以训练表现出色的生物医学MRC模型。我们通过将生物ADAPT-MRC与三种广泛使用的基准生物医学MRC数据集进行比较，从而广泛评估了生物ADAPT-MRC的性能-Bioasq-7B，BioASQ-8B和BioASQ-9B。我们的结果表明，如果不使用来自生物医学领域的任何合成或人类通知的数据，Bioadapt-MRC可以在这些数据集中实现最先进的性能。可用性：bioadapt-MRC可作为开放源项目免费获得，\ url {https://github.com/mmahbub/bioadapt-mrc}。

分布式训练是通过将任务分配到多个NPU（例如GPU/TPU）来减少DNN训练时间的解决方案。但是，分布式培训增加了NPU之间的通信开销，以使梯度和/或激活同步，具体取决于并行化策略。在用于大规模培训的下一代平台中，NPU将通过具有多种多样的异质带宽的多维网络连接。这项工作确定了一个迫在眉睫的挑战，即如果我们利用日程安排技术来使整个系统进行集体沟通，使所有网络维度保持繁忙并最大化网络BW。我们提出了Themis，这是一种新颖的集体调度方案，该方案动态调度集体（分为块）以平衡各个维度的通信负载，从而进一步改善了网络BW利用率。我们的结果表明，平均而言，Themis可以将单个全减还器的网络BW利用提高1.72倍（2.70倍），并改善实际工作负载的端到端训练迭代迭代迭代迭代迭代性能，例如RESNET-152，GNMT ，DLRM和Transformer-1T分别为1.49倍（最大2.25倍），1.30倍（1.78倍），1.30x（最大1.77倍）和1.25X（最大1.53倍）。

Recent work has shown that fine-tuning large pre-trained language models on a collection of tasks described via instructions, a.k.a. instruction-tuning, improves their zero and few-shot generalization to unseen tasks. However, there is a limited understanding of the performance trade-offs of different decisions made during the instruction-tuning process. These decisions include the scale and diversity of the instruction-tuning benchmark, different task sampling strategies, fine-tuning with and without demonstrations, training using specialized datasets for reasoning and dialogue, and finally, the fine-tuning objectives themselves. In this paper, we characterize the effect of instruction-tuning decisions on downstream task performance when scaling both model and benchmark sizes. To this end, we create OPT-IML Bench: a large benchmark for Instruction Meta-Learning (IML) of 2000 NLP tasks consolidated into task categories from 8 existing benchmarks, and prepare an evaluation framework to measure three types of model generalizations: to tasks from fully held-out categories, to held-out tasks from seen categories, and to held-out instances from seen tasks. Through the lens of this framework, we first present insights about instruction-tuning decisions as applied to OPT-30B and further exploit these insights to train OPT-IML 30B and 175B, which are instruction-tuned versions of OPT. OPT-IML demonstrates all three generalization abilities at both scales on four different evaluation benchmarks with diverse tasks and input formats -- PromptSource, FLAN, Super-NaturalInstructions, and UnifiedSKG. Not only does it significantly outperform OPT on all benchmarks but is also highly competitive with existing models fine-tuned on each specific benchmark. We release OPT-IML at both scales, together with the OPT-IML Bench evaluation framework.

Legal contracts, such as employment or lease agreements, are important documents as they govern the obligations and entitlements of the various contracting parties. However, these documents are typically long and written in legalese resulting in lots of manual hours spent in understanding them. In this paper, we address the task of summarizing legal contracts for each of the contracting parties, to enable faster reviewing and improved understanding of them. Specifically, we collect a dataset consisting of pairwise importance comparison annotations by legal experts for ~293K sentence pairs from lease agreements. We propose a novel extractive summarization system to automatically produce a summary consisting of the most important obligations, entitlements, and prohibitions in a contract. It consists of two modules: (1) a content categorize to identify sentences containing each of the categories (i.e., obligation, entitlement, and prohibition) for a party, and (2) an importance ranker to compare the importance among sentences of each category for a party to obtain a ranked list. The final summary is produced by selecting the most important sentences of a category for each of the parties. We demonstrate the effectiveness of our proposed system by comparing it against several text ranking baselines via automatic and human evaluation.

Project Loon is a Google initiated research project from the Google X Lab. The project focuses on providing remote internet access and network connectivity. The connectivity is established in vertical and horizontal space; vertical connectivity between Google Access Point (GAP) and the balloons, and between balloons and antennas installed at land; horizontal connectivity is between the balloons. This research focuses on the connectivity between the balloons in a mesh network. The proposal focuses on implementing graphical methods like convex hull with adhoc communication protocols. The proposed protocol includes content-based multicasting using angular sector division rather than grids, along with dynamic core-based mesh protocol defining certain core active nodes and passive nodes forming the convex hull. The transmission (multicasting and broadcasting) between the nodes will be evaluated using the link probability defining the probability of the link between two nodes failing. Based on the link probability and node features, best path between transmitting and receiver nodes will be evaluated.

Climate change, population growth, and water scarcity present unprecedented challenges for agriculture. This project aims to forecast soil moisture using domain knowledge and machine learning for crop management decisions that enable sustainable farming. Traditional methods for predicting hydrological response features require significant computational time and expertise. Recent work has implemented machine learning models as a tool for forecasting hydrological response features, but these models neglect a crucial component of traditional hydrological modeling that spatially close units can have vastly different hydrological responses. In traditional hydrological modeling, units with similar hydrological properties are grouped together and share model parameters regardless of their spatial proximity. Inspired by this domain knowledge, we have constructed a novel domain-inspired temporal graph convolution neural network. Our approach involves clustering units based on time-varying hydrological properties, constructing graph topologies for each cluster, and forecasting soil moisture using graph convolutions and a gated recurrent neural network. We have trained, validated, and tested our method on field-scale time series data consisting of approximately 99,000 hydrological response units spanning 40 years in a case study in northeastern United States. Comparison with existing models illustrates the effectiveness of using domain-inspired clustering with time series graph neural networks. The framework is being deployed as part of a pro bono social impact program. The trained models are being deployed on small-holding farms in central Texas.

ML-based motion planning is a promising approach to produce agents that exhibit complex behaviors, and automatically adapt to novel environments. In the context of autonomous driving, it is common to treat all available training data equally. However, this approach produces agents that do not perform robustly in safety-critical settings, an issue that cannot be addressed by simply adding more data to the training set - we show that an agent trained using only a 10% subset of the data performs just as well as an agent trained on the entire dataset. We present a method to predict the inherent difficulty of a driving situation given data collected from a fleet of autonomous vehicles deployed on public roads. We then demonstrate that this difficulty score can be used in a zero-shot transfer to generate curricula for an imitation-learning based planning agent. Compared to training on the entire unbiased training dataset, we show that prioritizing difficult driving scenarios both reduces collisions by 15% and increases route adherence by 14% in closed-loop evaluation, all while using only 10% of the training data.

We study politeness phenomena in nine typologically diverse languages. Politeness is an important facet of communication and is sometimes argued to be cultural-specific, yet existing computational linguistic study is limited to English. We create TyDiP, a dataset containing three-way politeness annotations for 500 examples in each language, totaling 4.5K examples. We evaluate how well multilingual models can identify politeness levels -- they show a fairly robust zero-shot transfer ability, yet fall short of estimated human accuracy significantly. We further study mapping the English politeness strategy lexicon into nine languages via automatic translation and lexicon induction, analyzing whether each strategy's impact stays consistent across languages. Lastly, we empirically study the complicated relationship between formality and politeness through transfer experiments. We hope our dataset will support various research questions and applications, from evaluating multilingual models to constructing polite multilingual agents.

We are interested in neurosymbolic systems consisting of a high-level symbolic layer for explainable prediction in terms of human-intelligible concepts; and a low-level neural layer for extracting symbols required to generate the symbolic explanation. Real data is often imperfect meaning that even if the symbolic theory remains unchanged, we may still need to address the problem of mapping raw data to high-level symbols, each time there is a change in the data acquisition environment or equipment. Manual (re-)annotation of the raw data each time this happens is laborious and expensive; and automated labelling methods are often imperfect, especially for complex problems. NEUROLOG proposed the use of a semantic loss function that allows an existing feature-based symbolic model to guide the extraction of feature-values from raw data, using `abduction'. However, the experiments demonstrating the use of semantic loss through abduction appear to rely heavily on a domain-specific pre-processing step that enables a prior delineation of feature locations in the raw data. We examine the use of semantic loss in domains where such pre-processing is not possible, or is not obvious. We show that without any prior information about the features, the NEUROLOG approach can continue to predict accurately even with substantially incorrect feature predictions. We show also that prior information about the features in the form of even imperfect pre-training can help correct this situation. These findings are replicated on the original problem considered by NEUROLOG, without the use of feature-delineation. This suggests that symbolic explanations constructed for data in a domain could be re-used in a related domain, by `feature-adaptation' of pre-trained neural extractors using the semantic loss function constrained by abductive feedback.

Breaking down a document or a conversation into multiple contiguous segments based on its semantic structure is an important and challenging problem in NLP, which can assist many downstream tasks. However, current works on topic segmentation often focus on segmentation of structured texts. In this paper, we comprehensively analyze the generalization capabilities of state-of-the-art topic segmentation models on unstructured texts. We find that: (a) Current strategies of pre-training on a large corpus of structured text such as Wiki-727K do not help in transferability to unstructured texts. (b) Training from scratch with only a relatively small-sized dataset of the target unstructured domain improves the segmentation results by a significant margin.